Skip to content

Getting detected by cloudflare #1388

Open
@vndhote

Description

@vndhote

undetected chromedriver worked well till yesterday but now, cloudflare improved and the chromedriver is not bypassing cloudflare. I have attached the screenshot of it. cloudflare is just looping the captcha when selenium is running. When I close it, the website loads.
image
This is my code snippet
image
It works fine when the script isn't running.
I also tried changing the binary to chrome rather than brave, but the issue still persists

Activity

zakio0403

zakio0403 commented on Jul 11, 2023

@zakio0403

I'm on the same spot, was working fine all this month and suddenly today it doesn't work

MrKhosrojerdi

MrKhosrojerdi commented on Jul 11, 2023

@MrKhosrojerdi

I have the same problem as you

vndhote

vndhote commented on Jul 11, 2023

@vndhote
Author

I'm on the same spot, was working fine all this month and suddenly today it doesn't work

I guess it gonna take some time to improve the logics in undetected chrome driver to be undetectable. Anyone having solution to this kindly send me friend request and dm me on discord
My discord Id : vivek9646

sh-erfan

sh-erfan commented on Jul 11, 2023

@sh-erfan

Same issue for me. Crawling 24 hours a day during last month, I got stuck a few hours ago.
IMPORTANT: When I close my bot and browser is still open, bypass is successful. This is strange for me.

benkrej

benkrej commented on Jul 11, 2023

@benkrej

Apparently that's an issue with the cloudflare captchas. I found that it works when dev tools are open. Adding the following line resolved the issue for me.
chrome_options.add_argument("--auto-open-devtools-for-tabs") # automatically open dev tools on every new tab
(This is a not a pretty solution ofc, I had no time to dive deeper yet)

digitalnomad91

digitalnomad91 commented on Jul 11, 2023

@digitalnomad91

I tried adding the dev-tools option @benkrej but I'm just getting the word "None":

options.add_argument( '--headless' )
options.add_argument( '--auto-open-devtools-for-tabs' )
chrome = uc.Chrome( options = options )
message = chrome.get( 'http://bscscan.com/contractsVerified' )
#chrome.save_screenshot( 'datadome_undetected_webddriver.png' )

print(message)

benkrej

benkrej commented on Jul 11, 2023

@benkrej

You will have to scrape and structure the data yourself, driver.get returns None. I suggest you to look into the coding examples.

salvecom333

salvecom333 commented on Jul 11, 2023

@salvecom333

Apperently that's an issue with the cloudflare captchas. I found that it works when dev tools are open. Adding the following line resolved the issue for me. chrome_options.add_argument("--auto-open-devtools-for-tabs") # automatically open dev tools on every new tab (This is a not a pretty solution ofc, I had no time to dive deeper yet)

god bless you man!

digitalnomad91

digitalnomad91 commented on Jul 11, 2023

@digitalnomad91

It worked, thank you!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

punkerpunker

punkerpunker commented on Jul 11, 2023

@punkerpunker

chrome_options.add_argument("--auto-open-devtools-for-tabs") # automatically open dev tools on every new tab

That worked mate, thanks!

Btw anyone knows how to close it afterwards? Site-specific protections detect me because it's opened.

I've tried action chains with Shift + Ctrl + i, but didn't succeed

jacobprice808

jacobprice808 commented on Jul 12, 2023

@jacobprice808

Same issue for me. Crawling 24 hours a day during last month, I got stuck a few hours ago. IMPORTANT: When I close my bot and browser is still open, bypass is successful. This is strange for me.

Does this not indicate that Cloudflare is perhaps tracking mouse movements now?

vndhote

vndhote commented on Jul 12, 2023

@vndhote
Author

Same issue for me. Crawling 24 hours a day during last month, I got stuck a few hours ago. IMPORTANT: When I close my bot and browser is still open, bypass is successful. This is strange for me.

Does this not indicate that Cloudflare is perhaps tracking mouse movements now?

Um.. like I tried using action chains to make it look more natural. I don't think so it may be mouse movements. It is possible that it's detecting selenium.

36 remaining items

baptx

baptx commented on Jul 26, 2023

@baptx

@NCLnclNCL Do you know which JavaScript code they can use to detect this?

benkrej

benkrej commented on Jul 26, 2023

@benkrej

@ultrafunkamsterdam is working on it as it seems so it may be worth to wait for his release.
#1420

baptx

baptx commented on Aug 9, 2023

@baptx

@benkrej The workaround with the tabs switch is still needed with the new version 3.5.2. I also have another issue: #1455.

benkrej

benkrej commented on Aug 9, 2023

@benkrej

You have to find a workaround yourself everything published is getting patched very soon.

baptx

baptx commented on Aug 9, 2023

@baptx

@benkrej Maybe there are things they cannot patch. If I find a workaround, I think I will share it.

benkrej

benkrej commented on Aug 9, 2023

@benkrej

Maybe it would be good to approach it another way and try to fix uc so it won't get detected in the first place without any temporary solutions. Would be awesome to have a chromedriver again like it was for the longest time!

baptx

baptx commented on Aug 10, 2023

@baptx

@benkrej I found a workaround: #1455 (comment)
I hope I did not make a mistake by sharing it :D

benkrej

benkrej commented on Aug 10, 2023

@benkrej

@baptx Works only with visible challenges some sites have no visible challenge and therefore are still stuck in the infinite loop. Glad you found a way that works for you!

miha1llumi

miha1llumi commented on Aug 28, 2023

@miha1llumi

I found out that if you open a new tab with js it passes cloudflare captchas. You can close the old tab and use the new one.

driver.execute_script('''window.open("http://nowsecure.nl","_blank");''') # open page in new tab
time.sleep(5) # wait until page has loaded
driver.switch_to.window(window_name=driver.window_handles[0])   # switch to first tab
driver.close() # close first tab
driver.switch_to.window(window_name=driver.window_handles[0] )  # switch back to new tab
time.sleep(2)
driver.get("https://google.com")
time.sleep(2)
driver.get("https://nowsecure.nl") # this should pass cloudflare captchas now

thanks, bro. Everything works.

iprahka

iprahka commented on Oct 2, 2023

@iprahka

I found out that if you open a new tab with js it passes cloudflare captchas. You can close the old tab and use the new one.

driver.execute_script('''window.open("http://nowsecure.nl","_blank");''') # open page in new tab
time.sleep(5) # wait until page has loaded
driver.switch_to.window(window_name=driver.window_handles[0])   # switch to first tab
driver.close() # close first tab
driver.switch_to.window(window_name=driver.window_handles[0] )  # switch back to new tab
time.sleep(2)
driver.get("https://google.com")
time.sleep(2)
driver.get("https://nowsecure.nl") # this should pass cloudflare captchas now

Youre absolutely right, but at the the same time I just tried to comment out the '--headless' line and it works perfect too without any long loading

Here's my working code version:

from selenium import webdriver
import time
from config import proxy_ip, proxy_port # you can remove it and the option below

options = webdriver.ChromeOptions()
# options.add_argument("--headless")
options.add_argument(f"--proxy-server={proxy_ip}:{proxy_port}")
options.add_argument('--user-data-dir=/home/iprahka/.config/google-chrome/Default')

driver = undetected_chromedriver.Chrome(options)

driver.get('https://www.nowsecure.nl')


time.sleep(99)
Ahmed-Ashraf-Khalil

Ahmed-Ashraf-Khalil commented on Nov 23, 2023

@Ahmed-Ashraf-Khalil

Screenshot 2023-11-23 034202

i saw this tool online garantee cloudflare verfication to be solved

cons:
the documentation is in chinese and couldn't link it with selenium.

n0nameNPC

n0nameNPC commented on May 6, 2024

@n0nameNPC

Apperently that's an issue with the cloudflare captchas. I found that it works when dev tools are open. Adding the following line resolved the issue for me. chrome_options.add_argument("--auto-open-devtools-for-tabs") # automatically open dev tools on every new tab (This is a not a pretty solution ofc, I had no time to dive deeper yet)

I tried your solutions, this and the new one and it works on my local machine. However it cannot seem to work on AWS ec2 or Digitalocean droplet. Do you have any pointers where to look to solve this?

are you using package for rendering ? For example: Xvfb

Ghxst

Ghxst commented on May 31, 2024

@Ghxst

Apperently that's an issue with the cloudflare captchas. I found that it works when dev tools are open. Adding the following line resolved the issue for me. chrome_options.add_argument("--auto-open-devtools-for-tabs") # automatically open dev tools on every new tab (This is a not a pretty solution ofc, I had no time to dive deeper yet)

I tried your solutions, this and the new one and it works on my local machine. However it cannot seem to work on AWS ec2 or Digitalocean droplet. Do you have any pointers where to look to solve this?

are you using package for rendering ? For example: Xvfb

I am, I can relate to having issues automating my browsers whenever the system is headless and I rely on Xfvb.
Any ideas why or tips to avoid detection of headless systems?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

      Participants

      @digitalnomad91@baptx@sh-erfan@mutiinsani@zakio0403

      Issue actions

        Getting detected by cloudflare · Issue #1388 · ultrafunkamsterdam/undetected-chromedriver